This disclosure relates to network monitoring.
The prevalence and accessibility of computer networks requires security measures to protect valuable information. An enterprise, for example, can implement such security measures by use of a layered security system. Such a layered security system can be implemented at the network edge of the enterprise, e.g., firewalls, gateway security agents, etc. Additionally, a layered security system can also include security processes and agents that are implemented throughout the enterprises, e.g., malware detection software on each computer device within the enterprise, content filtering software, content monitoring software, etc.
Existing methods of detecting malicious code outbreaks in computer networks are based on measuring a set of parameters and checking whether the parameters exceed a pre-defined threshold value. When this occurs malicious code is inferred. For example, the measurement parameters used are based on the number of e-mail messages sent a) with the same subject, b) with executable attachments, c) with identical file attachments, and d) with executable attachments of the same type. Another measure used is e-mail throughput, which is the product of the number of messages and the size of the message measured per unit quantum of time. Another technique is based on the number of e-mail messages generated in response to the arrival of an e-mail message. If the number of generated e-mails exceed a threshold, then a virus outbreak may be inferred.
Advancements in malware detection technology and malware propagation techniques provide an opportunity for better outbreak detection systems. For instance, a recent advancement is the use of ABM (Application Behavior Monitoring) to signal the presence of malware by monitoring abnormal patterns of events at each host system, allowing hosts to raise alarms for the events (including false alarms), aggregating alarms from multiple clients in steps of progressively increasing threshold levels. Examples of such events are deleting files, receiving infecting files, modifying registry keys, creating auto-run configurations, creating and modifying file-association registry keys, creating registry markers, creating shared folders, reading passwords, recording keyboard events, gathering email addresses, sending system related information, etc. However, such monitoring signals can be obtained only from infected hosts.
Other malware detection systems employ a distributed collection of virus scanning software coupled with an agent to report abnormal results to a centralized server. These results are progressively aggregated to create an outbreak alert system. The system has the disadvantage that only existing viruses can be detected through signature based scanning. Another disadvantage is the inability of virus scanning software from different vendors to inter-operate.
Malware detection engines that work on content inspection alone have fewer parameters to detect abnormal activity. In a host based malware detection system as seen in prior art, complete system activity is captured by the malware engine. Host based systems or collections of them cannot be used as they cannot handle large number of transactions even with a large collection of hosts. Secondly, execution of subscriber content may affect the privacy and confidentiality of the subscriber. Additionally, advanced malware can employ techniques such as root kits to hide from host based anti-virus systems altogether, making any statistical analysis by such systems meaningless.